1. Field of the Invention
The present invention relates to a method of calibrating the shape or the position and orientation of an object on a space, parameters of a camera itself, or the like, based on a captured image.
2. Description of the Related Art
There is known a method of analyzing an image that captures a marker or pattern including a characteristic point whose coordinate value is given in a three-dimensional (3D) space, and calculating the shape or the position and orientation of an object allocated on the 3D space, the intrinsic and extrinsic parameters of the camera itself, or the like. Note that the marker and parameter include not only ones which are artificially allocated but also natural features.
For example, in “Roger Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses”, IEEE Journal of Robotics and Automation, Vol. RA-3, No. 4, pp. 323-344, August 1987” (hereinafter “reference Y. Tsai”), Japanese Patent Laid-Open No. 06-137840, and Japanese Patent Laid-Open No. 2001-325069 (corresponding US 2001/0010514 A1), a camera calibration algorithm for calibrating camera intrinsic parameters or camera extrinsic parameters (hereinafter “camera parameters”) is disclosed. The camera calibration algorithm of this type estimates camera parameters by generating a teaching point image by capturing an image including a plurality of teaching points which are allocated on a 3D space coordinate system and have given coordinate values, and using the space coordinate values and image coordinate values of the teaching points.
Also, in “Gregory Baratoff, Alexander Neubeck, and Holger Regenbrecht: “Interactive Multi-Marker Calibration For Augmented Reality Applications”, Proceedings ISMAR2002, pp. 107-116, 2002” (hereinafter “reference Baratoff et al.”), the positions and orientations of a plurality of markers on the 3D space are calibrated by capturing images of markers serving as indices used for alignment of mixed reality (MR) a plurality of times from various directions.
In this way, with the methods using an image of the marker or pattern allocated on the 3D space to calibrate the camera parameters or the shape or the position and orientation of an object on the space from the image, the input image that captures the pattern or marker influences the final precision.
For example, the method of reference Y. Tsai is distinguished in that the calibration precision is impaired when the pattern is captured from a direction normal to the pattern. Also, the method that improves the estimation precision of the camera parameters using a plurality of pattern images is distinguished in that the precision cannot be expected to improve if only images obtained by capturing the pattern from similar directions are used. With the marker calibration by the method of reference Baratoff et al., if images to be used do not include a given marker, the position and orientation of the marker cannot be calibrated, and the total calibration precision drops in some cases. Thus, when the user who does not know the calibration mechanism captures images of the pattern or marker, appropriate calibration may often be disabled.